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ON TENSOR THIRD L-FUNCTIONS OF AUTOMORPHIC 
REPRESENTATIONS OF GL„(Aj.) 

HEEKYOUNG HAHN 


Abstract. Langlands’ beyond endoscopy proposal for establishing functoriality motivates 
interesting and concrete problems in the representation theory of algebraic groups. We 
study these problems in a setting related to the Langlands L-functions ®^), where 

TT is a cuspidal automorphic representation of GL„(Af) where F is a global field. 

1. Introduction 

Let F be a number field and let Ap be adeles of F. Let G be a reductive group over F. 
Given a representation 

—GL„(C), 

Langlands’ functorial conjectures [LI] predict there should be a corresponding transfer of 
automorphic representations of G{Ap) to automorphic representations of GL„(Af). 

One can ask for a characterization of those automorphic representations in the image. By 
the conjectural Langlands correspondence, these should correspond to F-parameters 

(p : Ff — GL„(C) 

such that a GL„(C)-conjugate factors through ^G, where Cp is the conjectural Langlands 
group. From this optic, one is led to ask how one can detect such parameters. 

Let ^G be the Zariski closure of Im(^G) viewed as a reductive group over C. Then a 
theorem of Ghevalley states that there exists a representation GL„ —>■ GL(y) such that ^G 
is the stabilizer of a line in V, see [Mj. Moreover, one knows via a result of Larsen and Pink 
[LP] that if ^G is connected semisimple, then the dimensions dim(I/^^) for all representations 
V of GL„ characterize ^G up to conjugation. Thus the following definition is natural: 

Definition 1.1. Let G be an irreducible reductive subgroup of GL„. We say a representation 
r : GL„ —> GL(Vj.) detects G if G fixes a line in Vj,- 

Remark. If G is connected then r detects G if and only if it detects G'^®'’. 

The following conjecture is the crux of Langlands’ beyond endoscopy proposal [L2], which 
aims to prove Langlands functoriality in general: 
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Conjecture 1.2. Let n he a unitary cuspidal automorphic representation o/GL„(Ap’). If n 
is a functorial transfer from G, then L{s, i^^r ®x) has a pole at s = 1 for some character 
X e F^\Af whenever r detects ^G. 

Thus implicit in Langlands proposal is a very concrete question in algebraic group theory. 
Question 1.3. Given a representation 

r : GL„ —^ GL(K-) 

which algebraic subgroups of GL„ are detected by r? 

If r = Sym^, one knows that every irreducible reductive subgroup of GL„ detected by r 
is conjugate to a subgroup of GO„. Moreover, in this case the Conjecture 1.2 is proven by 
work of Arthur [A], work of Cogdell, Kim, Piatetski-Shapiro and Shahidi [CKPSS] and work 
of Ginzburg, Rallis and Soundry [GRS]. There is a similar statement for r = A^. 

Apart from these special cases, explicit results are hard to come by. We mention one case 
that was discussed in a recent paper of Getz and Klassen [GK]. Let 

RS : GL^ X GL^ -- GL^2 

be the representation induced by the usual tensor product. Then it is known that RS{GLm x 
GLm) is detected by Sym™ (see [GK] for instance). Moreover, the analytic properties of the 
relevant Langlands L-function is well understood via the Rankin-Selberg theory, although 
we do not know its automorphicity. It is important to note that even in this situation Getz 
and Klassen do not discuss whether there are any other maximal subgroups of GL^z that 
are detected by Sym"*. 

Remark. Let tti and 7^2 be cuspidal automorphic representations of GLm(Ai?). Then one 
knows that 

L(s,7ri X 712, RS) = L{s,7ii X 772 ). 

Here the function on the left is the Langlands L-function and the function on the right is 
the usual Rankin-Selberg L-function. 

In this paper, we examine subrepresentations of the representation 

(x)3 : GL„ —> GL„3. 

In some sense this is the easiest case to consider after r = Sym^ and r = A^. This coincides 
with the setting of [GK] when we take m = 3 and n = 9. In this case we can obtain much 
more information than what is proven in loc. cit (see Thereom 1.8 below). 

Our hrst result is a lower bound on SA(SLm) not to be detected by (x)^ with respect to the 
number of nonzero parts of a partition A. Here Sa is the usual Schur functor associated to 
partition A. 
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Theorem 1.4. Let X be a partition with £ ^ m nonzero parts. If m > 3£, then the repre¬ 
sentation (x)^ does not detect 

Theorem 1.4 leads us to ask whether is detected by (x)^ if £ < m < 3i. We 

consider only the simplest case where A = (k) (see §2 for notation). So Sa = Sym^. In this 
case, £ = 1 so we ought to study when m = 2 and m = 3. 

Theorem 1.5. Let 

G := Sym"-i(SL2) GL„. 

The representation Sym^ detects G if and only if n = 1 (mod 4). 

Theorem 1.5 admits a pleasant partition-theoretic interpretation. Recall the following 
partition function: For hxed j,k e N, let p{k,j,n) be the number of partitions of n into at 
most i parts, with largest part at most k. Then we have the following: 

Corollary 1.6. For any integer i ^ 1, one has 

(1.0.1) p{A£ - 2, 3, 6£ - 3) = p{A£ - 2, 3, Q£ - 4) 

and 

(1.0.2) p{U, 3, Q£) - p{4:i, 3, Q£ - 1) = 1. 

The proof of Corollary 1.6 relies on representation theory. It would be interesting to hnd 
combinatorial proofs of (1.0.1) and (1.0.2). A. J. Yee pointed out the author that one might 
deduce the identities by employing symmetries of the coefficients of Gaussian polynomials, 
but the details are not obvious. 

For the case m = 3, we set 

(1.0.3) n - + 

for k e N. Then Sym^(SL 3 ) is naturally an irreducible reductive subgroup of GL„. One has 
the following result: 

Theorem 1.7. Let n be given as in (1.0.3) and let 

G := Sym^(SL3) GL„. 

Then for any k > 0, the representation <SP detects G. 

Finally, we examine closely irreducible connected reductive subgroups of GLg that are 
detected by Sym^. They are completley classihed via the following theorem: 
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Theorem 1.8. Let G ^ GLg be an irredueible connected reductive subgroup of GLg. Then 
if the representation Sym^ detects G, then either Lie{G'^^'^) = si 2 , or a GLg-conjugate of G 
is contained in RS{GL^ x GL 3 ). 

Remark. It would be very interesting to investigate non-connected reductive subgroups G < 

GLg. 

We close the introduction by outlining the paper. In §2, we provide an upper bound on m 
such that §A(SLm) is detected by the representation in terms of the number of parts of A. 
In §3, we discuss when Sym^ detects Sym^(SL 2 ). As an application, we present a partition 
theoretic interpretation of Theorem 1.5. In §4, we prove Theorem 1.7 and hnally in §5, we 
prove Theorem 1.8. 


2. SCHUR FUNCTORS 

Let A be a partition with at most m parts written as 
(2.0.1) A = (Ai, A 2 ,.. •, Am), Ai ^ A 2 ^ ^ Am ^ 0, 

and let |A| be the number partitioned by A. 

For any m dimensional vector space V over C and any partition A of |A| with at most m 
parts as in (2.0.1), we can apply the Schur functor Sa to V to obtain a representation §A(tL) 
of GLm- It remains irreducible when restrict to SLm- 

By the Littlewood-Richardson formula (compare with [FH, Exercise 15.23] ), one knows 
the decomposition of a tensor product of any two irreducible representations of SLm, namely 

(2.0.2) Sa(R)(x)§^(R) = 0C'a^A(R). 

V 

Here the coefficient G\^v are given by the Littlewood-Richardson rule and the sum is over 
partition v of |A| + |/i|. 

Then one has the following result: 

Theorem 2.1. Let X be a partition with i ^ m nonzero parts. If m > 3£, then the repre¬ 
sentation (x)^ does not detect §A(SLm). 

Proof. Let A be a partition of |A| whose number of nonzero parts is £ ^ m. In other words, 
A can be written as 

A = (Ai ,..., Xe, 0,. , 0 ), Ai ^ A 2 ^ ^ A^ > 0 . 

m—£ times 

Let V be the standard representation of SLm(C). Gonsider 

§a(R)®' = Hom(§A(R)",SA(R)®SA(R)) 
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(2.0.3) = ©Caa. Hom(§A(^)",S.(^)), 

V 

where we employ the Littlewood-Richardson formula (2.0.2) and denote by §a(R)'^ the dual 
space of Sa(R). Therefore the partitions v indexing the tensor product of (2.0.3) are parti¬ 
tions of 2|A| with at most m parts. 

Let be the partition such that §a(R)'^ = Sav(R). Then it is easy to check that A'" is 
(2.0.4) A'^ = ( Ai,. , Ap Ai — Af, Ai — A^-i,..., Ai — A 2 ,0) 

m—i times 

(compare with [FH, Exercise 15.50]). Therefore A'^ must have at least m — £ nonzero parts. 

Notice that two partitions p and ji' with at most m parts determine the same representation 
of SLm if and only if there is an integer b such that 

^ = fi' + b, 

that is, = ^[ + b for all 1 < i ^ m. Thus it suffices to prove that if m > 3£, then 

HomsL^(§Av+fe(R),§A(R)®SA(R)) = 0 

for all 6 e Z such that A'^ + 5 is a partition (i.e. all of its entries are nonnegative). In fact, 
such a 6 is necessarily nonnegative by (2.0.4). Thus partitions of the form X'' + b always 
have at least m — £ nonzero parts. 

On the other hand, the Littlewood-Richardson rule forces the partition v indexing the 
decomposition in (2.0.2) to have at most 2£ nonzero parts. This is a well-known fact, but 
we will provide a quick proof. To prove this, one first recalls that the number of boxes in 
the hrst column of the Young diagram of a given partition denotes the number of nonzero 
parts of it. Therefore A will have £ boxes in the first column in its Young diagram. Now we 
have to hll this Young diagram of A by the Littlewood-Richardson rule to obtain the Young 
diagram for z/. Starting with £ boxes in the hrst column corresponding to A, one is allowed 
to add only as many boxes in the hrst column as the number of parts of A which is in turn 
£. Therefore the maximum number of boxes in the hrst column of the Young diagram of v 
is 2£ (see [FH, Appendix A] for example). 

Assuming m — £ > 2£, we conclude that the partition X'' + b can never appear in the 
decomposition of tensor product, hence we prove that 

HomsL.(§A(R)^S.(R)) = 0. 


This completes the proof. 


□ 
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3. The subgroup Sym^(SL 2 ) 


In this section, we only consider a special types of irreducible reductive subgroups of GL„, 
namely G := Sym^(SL 2 ). In fact, we provide a precise condition on k for G to be detected 
by the representation Sym^, which implies that it is detected by (x)^. 

Recall the Gaussian polynomial 


(3.0.1) 


a 

b 




(1 - ^“-^>+ 1 ) 


(1 - q){l - g2 )... (1 _ qb) 


and the plethysm decomposition for GL 2 , namely: 


Theorem 3.1 (Theorem 5.5 [Do]). Let diml/ = 2. Then there is an isomorphism 0 /GL 2 - 
representations 

[jk/2\ 

(3.0.2) Sym^'(SymV) ^ 0 (Sym^'^-'^'R (x) 


where N{j, k, w) is the coefficient of in the polynomial (1 — q) 


j + k 
k 


L Jq 

In particular if j = 2 in Theorem 3.1 and R is a vector space of characteristic zero such 
that dimR = 2 , then one has 


[k/2\ 

(3.0.3) Sym2(Sym^R) ^ 0 Sym^^’^^R 

w^O 

as SL 2 -representations (see [Do, Exercise 5.16], for instance). One should notice that some 
terms in (3.0.2) vanishes in the decomposition in (3.0.3). In the following theorem, we give 
a complete answer to the question of when the representation Sym^ detects the subgroup 
Sym”-^(SL 2 ) of GL„: 


Theorem 3.2. Let 

G := Sym"-i(SL 2 ) GL„. 

The representation Sym^ detects G if and only if n = 1 (mod 4). 


Proof. Let IR be a representation of SL 2 . Then it is well-known that one has an isomorphism 
of SL 2 representations 

pp ®3 ^ Hom(lR'^, VP (X) IR) = Hom(lR'^, Sym^lR) © Hom(lR", A ^IR), 

where IR^ denotes the dual space of IR. Gonsidering the highest weights, it is easy to see 
that 


Sym^lR Hom(lR", Sym^lR) 
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as SL 2 -representations. Let W = Sym^y, where y is a complex vector space with dim V = 2. 
Then 

Hom(iy'^, Sym^LL) = Hom((Sym^y)'^, Sym^(Sym^y)) 

= Hom(Sym^(y''),Sym2(SymV)) 

= Hom(SymV,Sym2(SymV)) 

[fc/2j 

^ 0 Hom(SymV,Sym 2 ^-^V), 

i=0 

where in the last equality we use the Plethysm decomposition formula (3.0.3) and the fact 
that V'^ = V for dimy = 2. By Shur’s lemma, 

(3.0.4) HomsL 2 (SymV,Sym^^-^V) ^ 1, 

and HomsL 2 (Sym^y, Sym^^^'^^y) = 1 if and only if A: = 2fc — 4L Therefore A: = 0 (mod 4). 
By taking k = n — 1, we obtain the result. □ 

One then obtains immediate partition theoretic interpretation of Theorem 3.2. It is well- 
known that the generating function of p{k,j, n) is the Gaussian polynomial 

(3.0.5) = 

n^O 

(see [St, Proposition 1.7.3], for instance). We deduce the following corollary: 

Corollary 3.3. For any integer i ^ 1, one has 

p{M, 3, Qi) - p{4i, 3, 6 £ - 1) = 1 

and 

p{U - 2, 3, - 3) = p{U - 2, 3, - 4). 


j + k 
k 


Proof. Note that when 3A: — 2w = 0 in the decomposition (3.0.2), one has only one 1- 
dimensional summand. We then use the fact (3.0.4) in the proof of Theorem 3.2 to deduce 


that 


N{3,k,3k/2) 


J1 if A: = 0 (mod 4), 

1 0 ii k = 2 (mod 4). 


On the other hand, by Theorem 3.1, since N(3, A:, 3k/2) denotes the coefficient of ^^^2 


polynomial {I — q) 


3 + k 
k 


, N{3, k, 3k/2) is equal to the coefficient of in 


3 -t- A: 
k 


<? 


<? 
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minus the coefficient of g 3 A ;/2 i 


3 + k 
k 


Using the partition interpretation of the 


L Aq 

Gausssian polynomial as in (3.0.5), we derive that 


p(fc,3,3fc/2) -p(A:,3,3A:/2- 1) 
which in turn completes the proof. 


J 1 if fc = 0 (mod 4) 

1 0 if /c = 2 (mod 4), 


□ 


4. The subgroup Sym^(SL 3 ) 

In this section, we closely study the reductive subgroups Sym^(SL 3 ) for A: > 0. 

Theorem 4.1. Let n = 

G := Sym'=(SL3) -- GL„ 

for k > t). Then the representation (x)^ detects G. 


Proof. Let V be the standard representation of SL 3 . For A: > 0, we consider 

(4.0.1) (SymV)®3 ^ Hom((SymV)", SymV (x) SymV), 

where (Sym^U)'' denotes the dual space of Sym^U. We wish to investigate to see if 

Hom((SymV)",SymV(x)SymV) 0. 

As a special case of Pieri’s formula (compare with [FH, Exercise 15.33] ), one has that 

k 

(4.0.2) SymV(x)SymV = 0 S(fc+,,fc_p(U). 

i=0 

Let A (resp. A^) be the partition corresponding to Sym^U (resp. (Sym^U)^). It is easy 
to check that if m = 3, we have A'^ = {k,k,0) (see [FH, Excercise 15.50] for example). In 
particular, A^ appears in the decomposition of (4.0.2) if one takes i = 0. □ 


5. Connected reductive subgroups of GLg detected by Sym^ 

Throughout this section, we assume that G < GLg is an irreducible connected reductive 
subgroup. We characterize those G detected by Sym^: 

Theorem 5.1. Let G ^ GLg be an irreducible connected reductive subgroup. If the repre¬ 
sentation Sym^ detects G, then either Lie(G'^®'’) = sl 2 , or a conjugate of G is eontained 
%n RS{GL 3 X GL 3 ). 
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Proof. Let 

p:G ^GLg 

be the natural representation. Recall that G = , where Zq is the center of G. Thus 

it suffices to consider the irreducible representation of with dimension 9. 

From the dimension table of irreducible representations in [FK], one knows that Lie(G'^®'') 
is either sh, sla x SI 3 , SO 3 x SO 3 or sog. 

The case Lie(G'^®'') = slg coincides with Theorem 3.2 taking n = 9. In the case where 
Lie(G‘^®'') is isomorphic to s [3 x 513 or SO 3 x 503 , it is clear that Lie(G‘^®'') is contained in a 
GLg-conjugate of Lie(i?S'(GL 3 x GL 3 )). It follows that < i?>S'(GL 3 x GL 3 ). On the other 
hand, since the representation p is irreducible, p{Zc) must be contained in the subgroup of 
scalar matrices, and hence G < RS{GL^ x GL 3 ). For the last case Lie(G'^®'') = sOg, we let 
V be the standard representation of sog. Then from the decomposition of Sym^R into the 
direct sum of Schur functors [FH, Excercise 19.21(ii)], one knows that there is no trivial 
representation occurring in the decomposition. This completes our proof. □ 

Lastly we want to note that it would be interesting to investigate irreducible non-connected 
reductive subgroups of GLg, since the rank is not so big. This would require hnite group 
theory and Glifford theory. 
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